Skip to content

fix(marketplace): record the commit an annotated tag points to - #3161

Open
Rodion Kazennov (nefayran) wants to merge 8 commits into
microsoft:mainfrom
nefayran:fix/3048-annotated-tag-sha
Open

Rodion Kazennov (nefayran) wants to merge 8 commits into
microsoft:mainfrom
nefayran:fix/3048-annotated-tag-sha

Conversation

@nefayran

@nefayran Rodion Kazennov (nefayran) commented Oct 3, 2026 •

Copy link
Copy Markdown

Description

git ls-remote lists an annotated or signed tag twice: the tag object under refs/tags/<name> and the commit it points to under refs/tags/<name>^{}. _parse_ls_remote_output in src/apm_cli/marketplace/ref_resolver.py dropped the ^{} line, so apm pack wrote the tag object as source.sha. A checkout of the tag lands on the commit, and installers that compare the two reject the pin, as in the caveman case in the issue.

Both parsers now read tags through tag_commit_shas in src/apm_cli/deps/git_remote_ops.py, which gives a tag the SHA from its ^{} line; that line adds no ref of its own. It is registered as the owner ls-remote-tag-commits, with the static guard transport-platform-ls-remote-tag-commits. Lightweight tags and branches keep their only SHA. A ref: that is a full SHA never reaches this parser, so exact pins are unchanged. resolve_ref_sha uses the marketplace parser and is only called with HEAD, which has no ^{} line.

Issue and approved scope

Issue: #3048

Human scope-approval comment: #3048 (comment)

This PR completes the issue. A marketplace.json packed before the fix keeps the tag object SHA until apm pack runs again.

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance / refactor

Testing

  • Tested locally
  • All existing tests pass
  • Added tests for new functionality (if applicable)

tests/unit/marketplace/test_ref_resolver.py: test_peeled_tag_skipped is now test_annotated_tag_takes_peeled_commit_sha and expects the commit SHA. New cases cover a ^{} line that comes before its tag line, a branch with a lightweight and an annotated tag, and a ^{} line with no tag line.

tests/unit/marketplace/test_annotated_tag_source_sha.py creates the tags with git in a local bare repository (LocalGitRepositoryFactory): annotated, lightweight, and SSH-signed (skipped when ssh-keygen is missing). It packs a marketplace with ref: v1.0.0, version: "~1.0.0" and ref: v1.1.0 entries, then clones the repository, checks out each source.ref and compares HEAD with source.sha, the same check an installer makes. With the old parser, six of the new tests fail. tests/unit/deps/test_git_remote_ops.py covers tag_commit_shas directly, and the guard has its case in tests/integration/test_architecture_owner_rule_mutations.py. On 20,000 random ls-remote outputs parse_ls_remote_output returns the same refs before and after the change.

Locally uv run pytest tests/unit tests/test_console.py passes (23585 passed, 42 skipped, 21 xfailed). Ruff check and format, the pylint R0801 duplication check, scripts/lint-auth-signals.sh and scripts/lint-architecture-boundaries.sh are clean.

Spec conformance (OpenAPM v0.1)

  • Spec edit: docs/src/content/docs/specs/openapm-v0.1.md updated
    (new/changed <a id="req-XXX"></a> anchor + prose + Appendix C
    row).
  • Manifest edit: docs/src/content/docs/specs/manifests/openapm-v0.1.requirements.yml
    updated.
  • Test edit: a @pytest.mark.req("req-XXX") test under
    tests/spec_conformance/ added or extended.
  • CONFORMANCE.{md,json} regenerated via
    uv run --extra dev python -m tests.spec_conformance.gen_statement
    and committed.
  • N/A -- this PR does not change OpenAPM-observable behaviour.

`git ls-remote` lists an annotated or signed tag twice: the tag object
under `refs/tags/<name>` and its commit under `refs/tags/<name>^{}`.
`_parse_ls_remote_output` dropped the `^{}` line, so `apm pack` wrote the
tag object as `source.sha`, and installers that check out the tag and
compare the result rejected the pin. The tag now takes the peeled SHA, as
`parse_ls_remote_output` in `deps/git_remote_ops.py` already does.
Lightweight tags and branches keep their only SHA.

Fixes microsoft#3048

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Peeled-tag interpretation duplicates an existing durable decision instead of routing through one canonical owner.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Fixes marketplace packing so annotated and signed tags record the checked-out commit SHA rather than the tag-object SHA.

Changes:

  • Handles peeled ^{} tag references.
  • Adds parser and local-repository regression tests.
  • Documents the fix in the changelog.
File Description
src/​apm_cli/​marketplace/​ref_resolver.py Applies peeled commit SHAs to tags.
tests/​unit/​marketplace/​test_ref_resolver.py Tests parser edge cases.
tests/​unit/​marketplace/​test_annotated_tag_source_sha.py Tests real tags and packed output.
CHANGELOG.md Records the fix.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/apm_cli/marketplace/ref_resolver.py Outdated
The marketplace parser had its own reading of peeled `^{}` records next
to the one in `deps/git_remote_ops.py`. `tag_commit_shas` in
`git_remote_ops.py` now decides which commit a tag record names, with
the annotated-tag security note moved along; `parse_ls_remote_output`
and the marketplace `_parse_ls_remote_output` both read tags through
it, and the marketplace module no longer mentions `^{}`.

The owner is registered as `ls-remote-tag-commits` with the guard
`transport-platform-ls-remote-tag-commits`: one definition of
`tag_commit_shas`, and no `"^{}"` literal in `src/apm_cli` outside the
owner. The guard has its mutation case, and `tag_commit_shas` has
direct unit tests.
Preserve contributor history and validate microsoft#3161 against current main 9fb5ce8.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f79e464c-180e-4be3-9056-0773aa679a1a
@danielmeppiel

Daniel Meppiel (danielmeppiel) commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

APM Review Panel: ship_now

Terminal delta synthesis for marketplace source.sha annotated-tag fix: production code unchanged after main integration, all six CI workflows green at ba05281, clean merge preview against later main.

panel-mode=delta; personas=apm-ceo

cc Rodion Kazennov (@nefayran) Sergio Sisternes (@sergio-sisternes-epam) -- a fresh advisory pass is ready for your review.

This terminal CEO pass evaluates the final published head ba05281 after a faithful main dependency integration that introduced no changes to production src/, tests/, architecture scripts, or conformance outputs. The prior three panel rounds -- a five-specialist full panel, a five-specialist delta, and a conformance delta -- converged on zero remaining in-scope findings. The initial full panel raised four bounded recommendations (shared peeled predicate extraction, explanatory record-order comment, exact-SHA pin fixture with mutation proof, and producer+packaged documentation); all four were committed and verified clean by five genuine delta specialists with zero new findings. The conformance delta's test-coverage-expert confirmed that req-rs-002/req-rs-017 parametrized cases close the resolver-coverage gap with mutation proof. The sole outstanding suggestion (PERF-1: non-tag-then-commit iteration in marketplace) was correctly declined as out of scope -- it would alter ref ordering and orphan-peeled handling for an unmeasured constant-factor change on an O(n) path dominated by network RTT.

All six standard technical CI workflows completed successfully at the actual published head ba05281: CI (38756371499), CodeQL (38756371395), Spec Conformance (38756371446), Merge Gate (38756371391), Docs (38756371429), NOTICE Drift (38756371368). Full GraphQL rollup shows 23 green/success/neutral/skipped checks with no pending or failed entries. Local evidence at the tested merged base ec2444f confirms 660 passed / 2 skipped in 230.68s, 129 requirements aligned, generated conformance files clean, seven lint gates passed, and all four mutation scenarios failed-as-expected then restored. Importantly, local lint and tests ran against merged base ec2444f, not the later main f65ac26 -- these bases are distinct and this synthesis does not claim otherwise. Fresh git merge-tree preview against later main f65ac26 is clean with tree aa4f65e17f matching the GitHub PR merge tree exactly (merge commit c6c15a5, parents f65/ba0), confirming no source or index conflicts from the later main advance (#3209 req-pl-014 repair). Owner-touch-report confirms exactly one shared decision surface (tag_commit_shas and its two parser consumers), consistent with the dual-guardrail architecture design. The PR is MERGEABLE per GitHub; mergeStateStatus remains BLOCKED despite green technical checks and clean mergeability; existing review and protection requirements remain for the maintainer.

Aligned with: Marketplace source.sha now identifies the checkout commit for annotated and signed tags; fail-closed annotated-tag signal preserved and centralized behind canonical owner with mutation-tested architecture lint. Packed source metadata correctly identifies the selected commit across lightweight, annotated, and signed tags; CONFORMANCE.json links the behavior to req-rs-002 and req-rs-017. Single-owner tag_commit_shas with registered architecture rule and executable mutation guard prevents duplicated peeled-ref interpretation across parsers. Contributor history preserved; existing Daniel/Sergio review ownership unchanged; worker cannot merge.

Recommendation

All full-panel and delta-panel findings have been folded and verified across three genuine panel rounds with zero remaining in-scope gaps. CI is green across all six workflows at the actual published head. Local evidence is comprehensive at the tested merged base, and clean merge preview against later main confirms no compatibility issue. The PR is ready for maintainer merge at existing review-request and branch-protection discretion.

Folded in this run

Copilot signals reviewed

  • 4173071626: LEGIT duplicate-authority concern, already resolved by contributor commit efbfe64186; the existing resolved thread remains untouched. Two classification rounds found no new item.

Deferred (out-of-scope follow-ups)

Regression-trap evidence (mutation-break gate)

  • Removed peeled overwrite plus static rule reporting: 8 tests failed, including real-git behavior and architecture guard tests.
  • Removed exact-SHA short circuit: the real-git pack fixture failed on the SHA-pinned entry.
  • Removed annotated identity signal: both conformance record-order cases failed.
  • Every mutation was restored before final validation.

Lint contract

All seven repository lint gates passed at ba052812c2060fc1ad70eb9a85108267d22d26a2, locally tested against merged base ec2444fd448f7fd5a945f275758402276e67fc00: Ruff check and format, YAML I/O, 2100-line guard, portable relative paths, pylint R0801, and auth boundaries. Architecture boundary lint also passed.

CI

NOTICE Drift Check=success https://github.com/microsoft/apm/actions/runs/38756371368; Spec conformance=success https://github.com/microsoft/apm/actions/runs/38756371446; Deploy Docs=success https://github.com/microsoft/apm/actions/runs/38756371429; CodeQL=success https://github.com/microsoft/apm/actions/runs/38756371395; CI=success https://github.com/microsoft/apm/actions/runs/38756371499; Merge Gate=success https://github.com/microsoft/apm/actions/runs/38756371391; local tested base ec2444f; later main ee08bb5 has clean local merge-preview tree 8770ba33ea10231738507cc454ca0eecd5d203ff; PR-changed code paths are unchanged; upstream narrative additions retained; 344 passed in 44.26s on the combined preview tree, with architecture/auth boundary lint passed. CI Lint checked out c6c15a5 against main f65ac26. These are deliberately distinguished from the later-main preview.

After the CEO synthesis, main advanced again to ee08bb59b9a7bce67fb4010e4af8652c4c53a7de. Its fresh local merge preview is clean; PR-changed code paths are unchanged, and upstream narrative additions are retained. The combined marketplace tree additionally passed 344 passed in 44.26s, architecture boundary lint, and auth-signal lint. The uncommitted preview merge was then discarded, preserving the published head. Actual CI checked out c6c15a5fdbf56ea49e4f4491773b1149bd831785 against main f65ac26356353304f6a333ee738636be7b9206b4; original exact-head local tests used ec2444fd448f7fd5a945f275758402276e67fc00. The later preview is not claimed as another CI run. No speculative re-push or compatibility repair was needed.

One CI recovery: add truthful conformance coverage; no waiver, bypass, or workflow change.

Functional and canonical-owner evidence

ba052812c2060fc1ad70eb9a85108267d22d26a2: 660 passed, 2 skipped across real-git pack/checkout, exact pins, both parsers, architecture tests and complete conformance suite. All 129 requirements align and generated artifacts are clean.

Version-2 exact-base/head canonical-owner verification passed for the one detected tag-commit decision, including both consumer paths and the full dual guardrail.

Mergeability status

PR head SHA CEO stance iters folds defers Copilot rounds CI mergeable mergeStateStatus notes
#3161 ba052812c2 ship_now 4 4 1 2 green MERGEABLE BLOCKED Existing reviewer requests retained; no merge performed

Convergence

4 iterations; two Copilot rounds; no unresolved in-scope finding. Existing author and reviewer ownership is unchanged. Ready for maintainer review.

This panel is advisory. It does not block merge. Re-apply the
panel-review label after addressing feedback to re-run.


Generated by autopilot-pr-merge-worker. This comment is AI-generated and may contain errors.

Address the microsoft#3161 panel follow-ups with a real-git exact-SHA fixture, one peeled-record predicate, explicit marketplace iteration intent, and concise producer recovery guidance. Preserve selected refs and contributor history. Mutation trials fail when peeled identity, its static guard, or the exact-pin short circuit is removed.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f79e464c-180e-4be3-9056-0773aa679a1a
Address the exact-head Spec conformance failure with executable existing req-rs-002 and req-rs-017 coverage. Both record orders must preserve the peeled commit and annotated-only candidate selection. The mutation loses annotated identity and both cases fail. No new normative requirement or waiver.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f79e464c-180e-4be3-9056-0773aa679a1a
Refresh microsoft#3161 for independently landed microsoft#3222 and earlier microsoft#3100 before final exact-head lint and functional validation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f79e464c-180e-4be3-9056-0773aa679a1a
Preserve microsoft#3237 linked-resource lifetime clarification alongside microsoft#3222 and microsoft#3100. No implementation or scope change to microsoft#3161.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f79e464c-180e-4be3-9056-0773aa679a1a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f79e464c-180e-4be3-9056-0773aa679a1a

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants